Analyzing and Comparing the Protection Quality of Security Enhanced Operating Systems

Host compromise is a serious computer security problem today. To better protect hosts, several Mandatory Access Control systems, such as Security Enhanced Linux (SELinux) and AppArmor, have been introduced. In this paper we propose an approach to analyze and compare the quality of protection offered by these different MAC systems. We introduce the notion of vulnerability surfaces under attack scenarios as the measurement of protection quality, and implement a tool called VulSAN for computing such vulnerability surfaces. In VulSAN, we encode security policies, system states, and system rules using logic programs. Given an attack scenario, VulSAN computes a host attack graph and the vulnerability surface. We apply our approach to compare SELinux and AppArmor policies in several Linux distributions and discuss the results. Our tool can also be used by Linux system administrators as a system hardening tool. Because of its ability to analyze SELinux as well as AppArmor policies, it can be used for most enterprise Linux distributions and home user distributions.